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Abstract 


This  study  compared  the  effects  of  caffeine  ingestion  on  target  detection  and  rifle 
marksmanship  during  live  and  simulated  firing.  Thirteen  male,  rifle-trained  Canadian  Forces 
(CF)  reservists  performed  a  total  of  1  familiarization  and  4  experimental  sessions  over  an  8 
day  period.  Familiarization  was  conducted  on  the  small  arms  trainer  (SAT),  while  the  4 
experimental  sessions  were  randomly  and  evenly  distributed  between  2  sessions  of  SAT  and 
Automated  Targetry  System  (ATS)  live  shooting.  Each  session  was  comprised  of  8  shooting 
sequences  one  hour  following  the  ingestion  of  300  mg  of  caffeine  or  placebo.  Shooting 
sequences  included  both  friend-foe  (FF)  and  vigilance  (VIG)  tasks  and  were  performed  in  the 
following  sequence:  two  FF  engagements  (4  minutes  each),  four  VIG  engagements  (30 
minutes  each),  and  two  FF  engagements  (4  minutes  each).  The  shooting  session  lasted 
approximately  2.5  hours  for  both  the  caffeine  and  placebo  trials.  Performance  measures 
during  the  shooting  session  included  friend-foe  discrimination,  and  marksmanship  accuracy 
and  precision.  Assessments  of  thermal  comfort  and  tiredness  preceded  and  followed  the 
shooting  session.  The  Environmental  Symptoms  Questionnaire  was  administered  pre-  and 
post-shooting,  while  the  Task  Load  Index  was  administered  post-shooting  only.  Blood  was 
sampled  immediately  prior  to  the  beginning  of  the  shooting  session  and  was  used  to  determine 
plasma  caffeine  levels.  Caffeine  resulted  in  lower  subjective  sensations  of  effort  and  an 
increased  initial  level  of  thirst.  Subjects  reported  feeling  colder  on  the  ATS,  and  experienced 
blurred  vision  and  lower  levels  on  wakefulness  on  the  SAT.  Marksmanship  performance  was 
substantially  poorer  in  the  SAT  and  adversely  affected  by  caffeine.  No  caffeine-induced 
performance  decrements  were  noted  on  the  ATS,  but  caffeine  resulted  in  a  greater  number  of 
shots  taken  on  both  the  SAT  and  ATS.  The  results  of  this  study  indicate  that  although 
consistencies  between  live  and  simulator  fire  were  not  established,  rectifying  several  key 
technological  constraints  would  help  realise  the  research  potential  of  the  SAT. 
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Resume 


Cette  etude  comparait  les  effets  de  l'ingestion  de  cafeine  sur  la  detection  des  cibles  et  l'adresse 
au  tir,  dans  le  cadre  de  tirs  reels  et  simules.  Treize  reservistes  des  Forces  canadiennes  (FC),  de 
sexe  masculin  et  entraines  au  tir,  se  sont  soumis  a  1  seance  de  familiarisation  ainsi  qu'a 
4  seances  experimentales,  sur  une  periode  de  8  jours.  La  seance  de  familiarisation  a  eu  lieu  sur 
le  simulateur  d'entrainement  au  tir  aux  armes  legeres  (SAT),  tandis  que  les  4  seances 
experimentales  ont  ete  reparties  au  hasard  et  egalement  entre  2  seances  sur  le  SAT  et  les 
seances  de  tir  reel  sur  le  systeme  de  ciblerie  automatique  (SCA).  Chaque  seance  comportait 
8  sequences  de  tir,  executees  une  heure  apres  l'ingestion  de  300  g  de  cafeine  ou  d'un  placebo. 
Les  sequences  de  tir  comprenaient  des  taches  ^identification  ami-ennemi  (IFF)  et  des  taches 
de  vigilance  (V),  et  etaient  executees  dans  l'ordre  suivant :  deux  engagements  IFF  (4  minutes 
chacun),  quatre  engagements  V  (30  minutes  chacun),  et  deux  engagements  IFF  (4  minutes 
chacun).  La  seance  de  tir  durait  environ  2,5  heures,  pour  les  essais  avec  cafeine  comme  pour 
ceux  avec  placebo.  Les  indicateurs  de  performance  utilises  durant  les  seances  de  tir  etaient 
l'identification  ami-ennemi  et  l'exactitude  et  la  precision  du  tir.  Ces  seances  de  tir  etaient 
precedees  et  suivies  par  une  evaluation  du  confort  thermique  et  de  la  fatigue  ainsi  que  par 
l'administration  du  questionnaire  sur  les  symptomes  lies  a  l'environnement  (Environmental 
Symptoms  Questionnaire,  ESQ);  on  calculait  egalement  l'indice  de  charge  de  travail  (Task 
Load  Index)  a  la  fin  des  seances.  Des  echantillons  de  sang  etaient  preleves  immediatement 
avant  le  debut  de  la  seance  de  tir  pour  determiner  les  teneurs  plasmatiques  en  cafeine.  La 
presence  de  cafeine  s'est  traduite  par  une  diminution  des  sensations  d'effort  subjectives  et  une 
hausse  du  degre  de  soif  initial.  Les  sujets  ont  declare  avoir  eu  plus  froid  lors  des  seances  SCA, 
et  avoir  eu  la  vue  brouillee  et  un  niveau  d'eveil  moindre  lors  des  seances  SAT.  La 
performance  au  tir  a  ete  nettement  inferieure  dans  les  seances  SAT  et  a  ete  perturbee  par  la 
cafeine.  On  n'a  observe  aucune  diminution  de  performance  induite  par  la  cafeine  dans  les 
seances  SCA,  mais  la  presence  de  la  substance  s'est  traduire  par  une  augmentation  du  nombre 
de  coups  tires  dans  les  deux  situations  (SAT  et  SCA).  Les  resultats  de  cette  etude  indiquent 
que,  malgre  le  manque  d'uniformite  entre  les  tirs  reels  et  simules,  la  correction  de  plusieurs 
problemes  techniques  majeurs  permettrait  de  realiser  le  plein  potentiel  du  SAT  pour  la 
recherche. 
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Executive  summary 


As  part  of  a  research  program  examining  the  effects  of  physiological  stressors  on  target 
detection  and  rifle  marksmanship  (TD&M),  an  investigation  into  the  effect  of  caffeine 
ingestion  on  TD&M  was  conducted  on  both  a  live-fire  range  as  well  as  an  advanced  small 
arms  trainer  (SAT).  Caffeine  has  been  shown  to  increase  both  alertness  [12]  and  improve 
physical  work  capacity  [4]  .  Consequently,  it  has  been  proposed  to  be  of  potential  benefit  to 
soldier  performance.  For  example,  caffeine  has  also  been  shown  to  improve  both  vigilance 
and  friend/foe  discrimination  tasks  without  adversely  affecting  marksmanship  [12;  13]. 

These  studies  were  conducted  using  the  Weaponeer  simulator,  and  have  not  been  verified  in 
live-fire  conditions  or  on  other  marksmanship  simulators.  In  response  to  these  questions,  a 
study  was  conducted  to  determine  the  effect  of  caffeine  consumption  on  TD&M  during  live- 
fire,  and  if  the  SAT  could  be  used  as  a  convenient,  cost-effective  facility  for  conducting 
marksmanship  research.  While  the  effect  of  caffeine  alone  on  TD&M  during  live-fire  is 
addressed  in  a  concurrent  paper  [3],  this  report  examines  the  validity  and  limitations  of 
performing  marksmanship  studies  in  the  SAT  as  compared  to  a  live-fire  range. 

Thirteen  rifle-trained  militaiy  reservists  from  the  41  Canadian  Brigade  Group  volunteered  to 
participate  in  this  study.  Experimental  sessions  were  conducted  on  the  Automated  Targetry 
System  (ATS)  and  the  SAT  (FATS  IV,  FATS  Inc.,  Suwanne  GA)  ranges  at  CFB  Edmonton, 
supported  by  members  of  the  ASU  Edmonton  Training  Support  Company.  Each  subject 
underwent  one  familiarization  session  on  the  SAT,  followed  by  4  experimental  sessions  over 
an  8-day  period.  Two  of  these  sessions  were  conducted  on  the  ATS  range,  while  the  other 
two  were  conducted  on  the  SAT.  Each  session  consisted  of  the  ingestion  of  two  gelatin 
capsules  containing  either  300  mg  of  caffeine  or  a  placebo,  approximately  1  hour  prior  to 
shooting.  Caffeine/placebo  administration  was  randomized  and  blinded  to  the  subject  and 
range  staff.  All  weapon  firing  was  done  using  a  standard  or  SAT  modified  C7A1  rifle  with 
C79  scope.  Targets  were  presented  in  two  friend/foe  (FF)  sequences  of  4  minutes  each,  two 
vigilance  (VIG)  sequences  of  1  hour  each,  followed  by  an  additional  two  FF  sequences  for  a 
total  session  time  of  approximately  2.5  h.  All  targets  consisted  of  standard  or  modified,  figure 
E  targets  at  a  range  of  200  m.  Targets  were  modified  in  the  FF  condition  only,  and  consisted 
of  a  black  and  white  diagonal  striping  pattern  which  direction  was  used  to  differentiate 
friendly  from  foe  targets.  Blood  samples  were  taken  prior  to  shooting  to  determine  plasma 
caffeine  levels,  and  subjective  rating  questionnaires  were  administered  prior  to  and  upon 
completion  of  the  sessions.  Marksmanship  was  evaluated  for  precision  and  accuracy  of  shot 
location.  All  subjects  underwent  the  same  session  conditions  and  shooting  sequences  in  a 
randomized  but  counterbalanced  order. 

Subjective  questionnaires  revealed  a  decrease  in  perceived  exertion  with  caffeine  on  both 
ATS  and  SAT  ranges,  while  overall  cold  sensation  was  greater  on  the  ATS.  This  was  not 
surprising  as  the  ambient  temperature  of  the  ATS  range  was  cooler  than  the  SAT.  Subjects 
reported  cold  hands  and  feet  on  the  ATS,  while  blurry  vision  and  a  decreased  wakefulness 
were  experienced  on  the  SAT.  Individuals  also  perceived  their  performance  to  be  lower  on 
the  SAT.  Caffeine  resulted  in  a  greater  initial  level  of  thirst  and  decreasing  levels  of  tiredness 
over  the  duration  of  an  experimental  session.  In  contrast,  tiredness  increased  with  time  in  the 
placebo  condition. 
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Analysis  of  the  FF  data  demonstrated  that  the  subjects  were  reasonably  able  to  discriminate 
between  the  friendly  and  foe  targets.  When  comparing  appropriate  (e.g.  firing  at  foes  or 
holding  fire  on  friendly  targets)  and  inappropriate  responses,  more  inappropriate  responses 
were  made  on  the  SAT.  Most  errors  were  made  during  the  first  FF  sequence,  and  there  was 
no  difference  in  the  subsequent  sequences.  SAT  performance  was  worse  that  ATS  with 
respect  to  number  of  targets  hit.  Only  54%  of  SAT  targets  engaged  were  hit,  while  87%  of 
ATS  targets  found  their  mark.  Again,  the  poorest  performance  occurred  in  the  initial 
sequence. 

As  with  FF,  fewer  targets  were  hit  on  the  ATS  range  during  VIG,  and  the  targets  hit  to  shots 
ratio  was  also  lower.  Caffeine  consumption  resulted  in  subjects  increasing  the  number  of 
shots  taken  during  a  sequence.  Shot  location  data  generated  in  the  SAT  were  deemed  to  be  of 
poor  quality  and  susceptible  to  technological  deficiencies.  As  a  result  only  data  from  4 
subjects  were  useable.  From  this  limited  data  set,  it  was  apparent  that  marksmanship  accuracy 
and  precision  were  significantly  worse  in  the  SAT.  Caffeine  did  not  influence  any 
marksmanship  measures  in  the  ATS,  but  decreased  performance  in  the  SAT. 

The  results  of  this  study  indicate  marksmanship  performance  in  the  SAT  is  markedly  different 
from  live-fire  resulting  in  poorer  performance  in  almost  every  index  measured.  Although 
previous  researchers  were  able  derive  simple  correlations  between  live  and  simulated  shooting 
scores,  the  failure  to  reproduce  the  effects  of  a  stressor  (caffeine)  between  the  two  ranges 
suggest  that  the  scope  of  conformity  between  live  and  simulator  fire  may  be  limited. 

Much  of  the  difference  between  the  ranges  may  be  attributed  to  the  ambient  environment  in 
which  they  are  situated,  subjective  biases  against  the  simulator,  poor  projected  image  quality, 
and  technical  deficiencies  of  the  SAT.  A  major  technological  concern  centered  on  the  fact 
that  the  system  software  occasionally  failed  to  register  the  zeroing  information  of  a  weapon  at 
the  beginning  of  a  target  sequence.  Consequently,  shot  location  would  have  been  biased  by 
an  unknown  factor.  As  a  result,  it  is  highly  probable  that  the  quality  of  performance  measures 
was  adversely  affected.  Additionally,  as  the  SAT  was  designed  to  be  a  training  system,  the 
data  reporting  capability  was  limited  and  tedious  to  extract.  This  resulted  in  either  a  failure  to 
capture  data  due  to  time  constraints,  or  a  substantial  interruption  to  the  continuity  of  an 
experimental  session. 

http://corpranet.toronto.drdc-rddc.gc.ca/corpranet/services/imag_photo/guide_tips/tips.htmA 
comparison  of  SAT  and  live-fire  marksmanship,  with  and  without  caffeine  consumption, 
revealed  significant  differences  in  subjective  perceptions  and  shooting  performance.  Many  of 
these  differences  may  be  attributed  to  the  ambient  environment  in  which  they  are  situated  and 
the  known  technological  deficiencies  of  the  SAT.  While  field  environmental  conditions  are 
difficult  to  control,  it  is  doubtful  that  they  had  as  great  an  impact  on  performance  as  the 
technical  problems  of  the  SAT.  With  the  subsequent  delivery  of  a  new  generation  of  SATs, 
visual  acuity  has  been  markedly  improved,  and  the  zeroing  issue  addressed.  These  system 
improvements,  coupled  with  an  enhanced  data  reporting  capability  and  the  development  of  a 
SAT  facility  which  provides  a  similar  level  of  arousal  as  a  live  range  should  enable  the  SAT 
to  realize  its  potential  as  a  cost-effective  and  flexible  marksmanship  research  platform. 


Keefe,  A.  A.,  Tikuisis,  P.  2003.  A  comparison  of  target  detection  and  rifle  marksmanship 
during  live  and  simulator  firing  with  and  without  caffeine  consumption.  DRDC  Toronto 
TR  2003-003.  Defencr  R&D  Canada  -  Toronto. 
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Sommaire 


Dans  le  cadre  d'un  programme  de  recherche  portant  sur  les  effets  des  facteurs  de  stress 
physiologiques  sur  la  detection  des  cibles  et  l'adresse  au  tir,  une  etude  sur  l'effet  de  l'ingestion 
de  cafeine  a  ete  realisee  sur  un  champ  de  tir  reel  ainsi  que  sur  un  simulateur  avance 
d'entrainement  au  tir  aux  armes  legeres  (Small  Arms  Trainer,  SAT).  II  est  apparu  que  la 
cafeine  augmentait  a  la  fois  la  vivacite  d’esprit  [12]  et  la  capacite  de  travail  physique  [4],  ce 
qui  donne  a  penser  que  la  cafeine  pourrait  eventuellement  avoir  un  effet  benefique  sur  la 
performance  des  soldats.  II  est  egalement  apparu  que  la  cafeine  ameliorait  la  vigilance  et 
l'identification  ami-ennemi,  sans  nuire  a  l'adresse  au  tir  [12;  13].  Ces  etudes  ont  ete  realisees  a 
l'aide  du  simulateur  Weaponeer,  et  n'ont  pas  ete  verifiees  en  conditions  de  tir  reel  ou  sur 
d'autres  simulateurs  de  tir.  En  consequence,  une  etude  a  ete  effectuee  pour  evaluer  l'effet  de  la 
consommation  de  cafeine  sur  la  detection  des  cibles  et  l'adresse  au  tir  en  conditions  de  tir  reel, 
et  pour  savoir  si  le  SAT  pouvait  servir  d'installation  commode  et  economique  pour  les 
recherches  sur  l'adresse  au  tir.  L'effet  de  la  cafeine  seule  sur  la  detection  des  cibles  et  l'adresse 
au  tir  en  conditions  de  tir  reel  fait  l'objet  d'un  article  publie  simultanement  [3];  nous  nous 
penchons  ici  sur  la  validite  et  les  limites  des  etudes  sur  l'adresse  au  tir  realisees  au  moyen  du 
SAT,  par  rapport  aux  etudes  menees  dans  un  champ  de  tir  reel. 

Treize  reservistes  entraines  au  tir  du  41e  Groupe-brigade  du  Canada  se  sont  portes  volontaires 
pour  cette  etude.  Des  seances  experimentales  ont  ete  organisees  sur  le  systeme  de  ciblerie 
automatique  (SCA)  et  le  simulateur  de  tir  aux  armes  legeres  (SAT,  FATS  IV,  FATS  Inc., 
Suwanne  GA)  a  la  BFC  d'Edmonton,  avec  l'appui  des  membres  de  la  compagnie  de  soutien  a 
l'instruction  de  l'USS  Edmonton.  Apres  une  seance  de  familiarisation  sur  SAT,  chaque  sujet  a 
ete  soumis  a  4  seances  experimentales  sur  une  periode  de  8  jours.  Deux  de  ces  seances  ont  eu 
lieu  sur  le  champ  de  tir  SCA,  et  les  deux  autres  sur  le  SAT.  A  chaque  seance,  les  sujets 
ingeraient  deux  gelules  de  gelatine  contenant  soit  300  mg  de  cafeine,  soit  un  placebo,  environ 
1  heures  avant  la  sequence  de  tir.  L'administration  de  la  cafeine  ou  du  placebo  se  faisait  au 
hasard  et  a  l'insu  des  sujets  et  du  personnel  du  champ  de  tir.  Tous  les  tirs  ont  ete  executes  a 
l'aide  de  fusils  C7A1  modifies,  avec  lunette  de  visee  C79.  Les  cibles  etaient  presentees  dans  le 
cadre  de  deux  sequences  d'identification  ami-ennemi  (IFF)  de  4  minutes  chacune  et  de  deux 
sequences  de  vigilance  (V)  d'une  heure  chacune,  suivies  de  deux  nouvelles  sequences  IFF, 
pour  une  seance  d'une  duree  totale  d'environ  2,5  heures.  Toutes  les  cibles  consistaient  en 
cibles-silhouettes  E  standard  ou  modifiees,  a  une  distance  de  200  m.  Les  cibles  n'etaient 
modifiees  qu'en  condition  IFF,  et  consistaient  en  une  surface  noir  et  blanc  rayee  en  diagonale, 
dont  la  direction  servait  a  differencier  la  nature  (ami  ou  ennemi).  Des  echantillons  de  sang  ont 
ete  preleves  avant  chaque  tir  afin  de  determiner  la  teneur  plasmatique  en  cafeine;  les  sujets 
devaient  en  outre  repondre  a  un  questionnaire  depreciation  subjective  avant  et  apres  les 
seances.  L'adresse  au  tir  etait  evaluee  sur  le  plan  de  la  precision  et  de  l'exactitude  du  tir.  Tous 
les  sujets  ont  ete  soumis  aux  memes  conditions  et  aux  memes  sequences  de  tir  dans  un  ordre 
aleatoire,  mais  equilibre. 

Les  questionnaires  depreciation  subjective  ont  mis  en  evidence  une  diminution  de  la  fatigue 
perque  chez  les  sujets  qui  avaient  pris  de  la  cafeine,  sur  le  champ  de  tir  SCA  et  sur  le  SAT, 
ainsi  qu'une  sensation  generate  de  ffoid  accrue  dans  le  champ  de  tir  SCA.  Ce  dernier  resultat 
n'est  guere  etonnant  etant  que  la  temperature  ambiante  du  champ  de  tir  SCA  etait 
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effectivement  plus  basse  que  sur  le  SAT.  Les  sujets  ont  declare  avoir  eu  froid  aux  mains  et 
aux  pieds  sur  le  champ  de  tir  SCA,  et  avoir  eu  la  vue  brouillee  et  un  niveau  d'eveil  moindre 
sur  le  SAT.  Les  sujets  ayant  ingere  de  la  cafeine  ont  declare  un  degre  de  soif  initial  plus 
prononce  et  une  fatigue  moindre  sur  la  duree  d'une  seance  experimentale,  alors  que  chez  les 
sujets  ayant  absorbe  un  placebo,  la  fatigue  a  augmente  avec  le  temps. 

L'analyse  des  donnees  IFF  a  revele  que  les  sujets  etaient  raisonnablement  en  mesure  de 
distinguer  entre  les  deux  types  de  cibles.  Pour  ce  qui  est  de  la  comparaison  des  reponses 
correctes  (p.  ex.  tirer  sur  les  cibles  ennemies  et  retenir  le  tir  sur  les  cibles  amies)  et 
incorrectes,  le  nombre  de  reponses  incorrectes  etait  plus  eleve  sur  le  SAT.  La  majorite  des 
erreurs  ont  ete  commises  durant  la  premiere  sequence  IFF;  on  n'a  plus  observe  de  difference 
lors  des  sequences  suivantes.  En  ce  qui  conceme  le  nombre  de  cibles  touchees,  la  performance 
etait  moins  bonne  sur  le  SAT  que  sur  le  champ  de  tir  SCA.  Seulement  54  %  des  cibles 
presentees  sur  le  SAT  ont  ete  touchees,  contre  87  %  des  cibles  SCA.  Encore  une  fois,  la 
mauvaise  performance  a  ete  observee  lors  de  la  sequence  initiale. 

Comme  dans  les  sequences  IFF,  le  nombre  de  cibles  touchees  lors  des  sequences  SCA  a  ete 
moins  eleve  dans  les  taches  de  vigilance,  et  le  rapport  entre  les  coups  au  but  et  les  coups  tires 
a  egalement  ete  plus  faible.  Les  sujets  qui  avaient  ingere  de  la  cafeine  ont  tire  plus  souvent 
pendant  les  sequences.  Les  donnees  sur  les  points  d'impact  produites  par  le  SAT  ont  ete 
jugees  de  mauvaise  qualite  et  sensibles  aux  deficiences  technologiques,  si  bien  qu'on  n'a  pu 
utiliser  que  les  donnees  de  4  sujets.  A  partir  de  cet  ensemble  limite  de  donnees,  il  est  apparu 
que  l'exactitude  et  la  precision  des  tirs  etaient  de  loin  inferieures  dans  les  seances  SAT.  La 
cafeine  n'a  eu  par  ailleurs  aucune  incidence  sur  les  indicateurs  de  l'adresse  au  tir  en  situation 
SCA,  mais  a  abaisse  la  performance  sur  le  SAT. 

Les  resultats  de  cette  etude  revelent  que  la  performance  au  tir  sur  le  SAT  differe  beaucoup  de 
celle  observee  en  conditions  de  tir  reel,  ce  qui  se  traduit  par  une  moins  bonne  performance 
pour  presque  tous  les  indices  mesurees.  Bien  que  des  chercheurs  aient  deja  reussi  a  deriver  des 
correlations  simples  entre  les  scores  de  tir  en  conditions  reelles  et  en  conditions  simulees,  le 
defaut  de  reproduire  les  effets  d'un  facteur  de  stress  (la  cafeine)  entre  les  deux  champs  de  tir 
donne  4  penser  que  l'ampleur  de  la  conformite  entre  le  tir  reel  et  simule  pourrait  etre  limitee. 

La  difference  entre  les  champs  de  tir  peut  en  grande  partie  etre  attribute  au  milieu  ambiant 
dans  lequel  ils  se  trouvent,  aux  prejuges  defavorables  a  l'egard  du  simulateur,  a  la  mauvaise 
qualite  des  images  projetees  et  aux  deficiences  techniques  du  SAT.  Un  probleme  technique 
important  venait  de  ce  que  le  logiciel  du  systeme  omettait  parfois  d'enregistrer  les 
informations  sur  le  zerotage  d'une  arme  au  debut  d'une  sequence  de  cible.  En  consequence,  les 
donnees  sur  le  point  d'impact  ont  sans  doute  ete  biaisees  par  un  facteur  inconnu.  II  est  done 
fort  probable  que  cela  ait  eu  une  incidence  negative  sur  la  qualite  des  indicateurs  de 
performance.  En  outre,  comme  le  SAT  a  ete  congu  comme  un  systeme  d'entrainement,  sa 
capacite  de  communiquer  des  donnees  etait  limitee  et  les  donnees  etaient  penibles  a  extraire, 
ce  qui  s'est  traduit  soit  par  le  defaut  d'extraire  les  donnees  a  cause  des  contraintes  temporelles, 
soit  par  une  interruption  importante  dans  la  continuity  d'une  seance  experimentale. 

Une  comparaison  de  l'adresse  au  tir  sur  le  SAT  et  en  conditions  reelles,  avec  et  sans 
consommation  de  cafeine,  a  mis  en  evidence  d'importantes  differences  dans  les  perceptions 
subjectives  et  la  performance  du  tir.  Nombre  de  ces  differences  pourraient  etre  attribuables  au 
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milieu  ambiant  dans  lequel  ont  lieu  les  seances  et  aux  deficiences  techniques  connues  du 
SAT.  Bien  que  les  conditions  ambiantes  du  champ  de  tir  soient  difficiles  a  controler,  on  peut 
douter  qu'elles  aient  eu  autant  d'impact  sur  la  performance  que  les  problemes  techniques  du 
SAT.  La  prochaine  generation  de  simulateurs  SAT  offfe  une  acuite  visuelle  nettement 
superieure,  et  le  probleme  du  zerotage  a  ete  resolu.  Ces  ameliorations,  alliees  a  une  meilleure 
capacite  de  communication  des  donnees  et  au  developpement  d'une  installation  SAT 
procurant  un  degre  de  stimulation  semblable  a  celui  d'un  champ  de  tir  reel,  devraient 
permettre  au  SAT  de  realiser  son  plein  potentiel  comme  plate-forme  souple  et  economique 
pour  la  recherche  sur  1'adresse  au  tir. 


Keefe,  A.  A.,  Tikuisis,  P.  2003.  A  comparison  of  target  detection  and  rifle  marksmanship 
during  live  and  simulator  firing  with  and  without  caffeine  consumption.  DRDC  Toronto 
TR  2003-003.  Defence  R&D  Canada  -  Toronto. 
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Introduction 


Simulators  are  increasingly  being  used  as  a  complement/surrogate  to  live  fire  practice  for  training 
soldiers  in  small  arms  handling  and  marksmanship.  Simulators  provide  the  benefit  of  an 
increasingly  realistic  shooting  experience  in  a  safe,  controlled,  and  cost  effective  manner.  A 
variety  of  training  scenarios  can  be  quickly  authored  and  immediate  performance  feedback  is 
provided  to  the  soldier.  In  more  sophisticated  systems,  detailed  information  regarding  aim  point, 
weapon  status,  and  handling  may  also  be  reported. 

Although  the  benefits  of  marksmanship  simulators  appear  to  be  self  evident,  the  transfer  of  skills 
learned  in  the  simulator  to  live-fire  applications  requires  validation.  Studies  addressing  this  issue 
have  investigated  the  Weaponeer  [16],  Engagement  Skills  Trainer  [6],  Multipurpose  Arcade 
Combat  Simulator  [19],  and  Small  Arms  Trainer  (SAT)  [1;  2;  5]  systems.  These  validation 
studies  have  focused  on  the  comparison  of  scores  on  standardized  weapon  qualification  tasks 
normally  conducted  on  a  live-fire  range.  In  general,  these  studies  have  found  a  positive 
correlation  between  live  and  simulated  firing  performance  with  a  large  variation  of  training  effect 
within  the  sample  population.  For  example,  White  [19]  found  that  those  with  poor  marksmanship 
ability  benefited  the  most  from  simulator  training,  while  the  average  and  highly  skilled  trainee  did 
not  benefit.  Also,  simulator  performance  scores  relative  to  live  fire  tend  to  worsen  with 
increasing  distance.  Gula  [5]  found  simulator  scores  to  be  higher  at  a  target  distance  of  100  m, 
while  live-fire  scores  were  better  at  200  m.  It  was  hypothesized  that  poor  resolution  of  the 
projected  images  and  inadequate  or  inappropriate  perceptual  cues  may  cause  greater  difficulty  in 
the  judgement  of  distances  of  the  simulator  targets.  This  suggests  a  possible  limitation  to  the 
validity  of  the  use  of  training  simulator  engagements  authored  across  a  wide  range  of  target 
distances. 

Recently,  researchers  have  used  simulators  in  an  effort  to  quantify  the  effect  of  physiological  and 
psychological  stressors  and/or  ergogenic  aids  on  marksmanship.  The  Weaponeer  training 
simulator  has  been  utilized  in  the  investigation  of  the  effect  of  numerous  stressors/aids  such  as 
heat  stress  [8],  protective  clothing  [10],  prolonged  vigilance  [11],  and  caffeine  [12;  13;  14]  on 
target  detection  and  marksmanship  (TD&M).  Tikuisis  et  al.  [18]  investigated  the  effect  of  heat 
and  cold  strain  on  TD&M  using  the  SAT.  These  studies  employed  novel  engagements  involving 
extended  periods  of  vigilance,  friend/foe  identification,  and  moving  targets.  Whether  the  live 
fire/simulator  relationships  determined  in  earlier  studies  apply  to  these  new  engagements  has  yet 
to  be  determined.  Adding  to  the  complexity  of  this  research  is  the  confounding  aspect  of  unique 
target  engagement  that  different  simulator  technologies  present.  There  has  not  been,  to  our 
knowledge,  a  cross  validation  study  comparing  the  effects  of  stressors/aids  between  various 
simulators. 

In  order  to  validate  marksmanship  simulators  as  surrogate  research  platforms  in  which  to 
determine  the  scope  of  their  applicability,  a  significant  relationship  between  the  performance  of 
live  and  simulated  fire  must  be  established  across  a  variety  of  target  engagements.  In  addition, 
any  effect  due  to  stressors  or  aids  on  live  marksmanship  performance  must  be  reproducible  in  the 
simulator  in  a  consistent  and  predictable  manner. 

In  an  effort  to  investigate  these  relationships,  a  study  was  devised  to  compare  TD&M 
performance  of  live  and  simulated  fire  during  engagements  involving  friend/foe  discrimination 
and  vigilance.  The  SAT  was  the  simulator  selected,  as  it  is  the  primary  rifle-training  simulator  of 
the  Canadian  Forces  (CF).  Participants  were  given  either  a  placebo  or  caffeine  (300  mg)  in  both 
live  and  simulator  conditions.  Caffeine  has  been  demonstrated  to  improve  target  detection  time, 
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while  not  affecting  marksmanship  [14].  Based  on  previous  literature  and  experience,  it  was 
hypothesized  that  TD&M  performance  would  be  similar  in  both  live  and  simulated  engagements, 
and  that  the  performance  enhancing  effects  of  caffeine  would  be  present  in  both  conditions  in  a 
predictable  and  consistent  manner. 
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Methodology 


Subjects 

Thirteen  rifle-trained  military  reservists  (age  range  and  mean  ±  SD:  18-49  and  28  ±  9  yr) 
belonging  to  units  within  the  41  Canadian  Brigade  Group,  Edmonton,  volunteered  to  participate 
in  this  study.  Subjects  underwent  a  medical  screening  and  anthropometric  measurements  (height: 
170  -  188  and  179.1  ±  4.8  cm;  weight:  68-114  and  83.1  ±  4.8  kg),  and  were  given  a  full 
explanation  of  all  risks,  procedures,  and  benefits  prior  to  giving  their  informed  consent  in 
accordance  with  protocol  approval  requirements  from  the  DRDC  Toronto  Human  Research 
Ethics  Committee. 

Subjects  were  asked  to  refrain  from  caffeine  (eg.,  coffee,  caffeinated  tea  and/or  soft  drinks, 
chocolate  bars)  and  exercise  for  12  h,  and  abstain  from  alcohol  for  24  h  prior  to  arrival  for  a  trial. 
Aside  from  these  exceptions,  subjects  were  asked  to  conduct  their  normal  daily  routine  during  the 
entire  experiment.  A  questionnaire  was  administered  to  provide  an  estimate  of  the  subject’s 
normal  daily  caffeine  and  nicotine  intake.  Tobacco  users  were  permitted  to  chew,  but  not  smoke, 
during  the  trials.  Potential  subjects  were  screened  regarding  the  use  of  any  medication,  over  the 
counter  cold  remedies,  or  antidepressants  that  may  have  precluded  their  participation. 


Experimental  trials 

Subjects  underwent  one  familiarization  session,  followed  by  4  experimental  sessions  over  an  8- 
day  period.  Since  all  subjects  were  familiar  with  live-fire  procedures,  the  familiarization  session 
was  only  conducted  on  the  SAT  (FATS  IV,  FATS  Inc.,  Suwanee,  GA).  This  session  consisted  of 
the  ingestion  of  300  mg  of  caffeine  and  approximately  2.5  h  of  rifle  shooting  at  target  scenarios 
identical  to  those  that  would  be  experienced  in  the  experimental  sessions. 

Two  of  the  experimental  sessions  consisted  of  live-fire  engagements  held  on  the  Automated 
Targetry  System  (ATS)  range  (ATA  Defence  Industries,  Camden,  TN),  while  the  other  two  were 
conducted  on  the  SAT,  both  located  at  CFB  Edmonton.  In  each  session,  subjects  were  given 
gelatine  capsules  containing  either  a  placebo  (P)  or  300  mg  caffeine,  hereafter  denoted  as  the  drug 
(D),  approximately  1  h  prior  to  shooting.  All  sessions  consisted  of  firing  a  standard  or  SAT- 
modified  C7A1  rifle,  equipped  with  a  C79  scope  at  pop-up  targets  at  a  range  of  200  m.  The  SAT 
rifle  is  modified  with  the  addition  of  weapon  function  sensors,  a  laser  device  in  the  barrel,  and  a 
compressed  air  hose  connected  to  the  forestock.  Targets  were  presented  in  two  friend/foe  (FF) 
sequences  of  4  min  each,  two  vigilance  (VIG)  sequences  of  1  h  each,  followed  by  two  more  FF 
sequences  for  a  total  session  time  of  approximately  2.5  h.  Subjective  ratings  questionnaires  were 
administered  prior  to  and  upon  completion  of  the  shooting  sessions.  All  subjects  underwent  the 
same  session  conditions  and  shooting  sequences  in  a  randomized  but  counterbalanced  order. 
Figure  1  gives  an  overview  of  an  experimental  session. 
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Figure  1.  Timeline  of  each  experimental  session;  see  text  for  explanation  of  abbreviations 


Trial  randomization 


Subjects  were  randomly  assigned  to  participate  in  either  a  morning  (A)  or  afternoon  (B)  group. 
Group  A  conducted  their  activities  between  0700h  and  1200h,  and  group  B  between  1300h  and 
1800h.  Within  these  groups,  subjects  were  further  randomized  such  that  on  each  trial  day,  one 
half  would  conduct  live  fire  (L)  on  the  ATS  range  while  the  other  half  would  be  engaged  with 
simulated  fire  (S)  on  the  SAT. 

Each  group  was  counterbalanced  with  respect  to  drug  vs.  placebo,  live  vs.  simulated  fire,  and 
shooting  sequences.  Four  different  but  equally  difficult  shooting  sequences  (a  to  d)  were 
implemented  for  both  FF  and  VIG  engagements.  Ideally,  16  subjects  were  required  to  complete 
the  balanced  design.  As  only  15  subjects  were  available  to  participate  in  the  study  and  two  of 
these  from  the  morning  group  withdrew  during  the  course  of  the  study,  an  imbalance  in  the 
experimental  design  was  unavoidable.  Table  1  details  the  resultant  randomization  matrix  and  its 
implementation. 

Table  1.  Randomization  of  trials  where  A  and  B  refer  to  the  am  and  pm  groups,  and  D  and  P  refer  to  the 
drug  and  placebo,  respectively.  The  hatched  and  grey  rows  indicate  subject  dropout,  and  incomplete 
design,  respectively.  The  lowercase  letters  indicate  shooting  sequence. 
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Drug  ingestion 

Each  trial  began  with  the  ingestion  of  two  gelatin  capsules  containing  either  a  placebo 
(Metamucil  ®)  or  300  mg  of  caffeine,  and  given  a  light  snack  consisting  of  a  muffin  and  300  mL 
of  fruit  juice.  The  fixed  dose  of  caffeine  resulted  in  a  range  from  4.4  to  2.6  mg*kg4  depending 
on  the  subject’s  body  mass,  and  is  equivalent  to  the  amount  consumed  in  2  to  4  cups  of  regular 
coffee. 

Blood  sampling 

Approximately  1  h  after  capsule  ingestion,  a  3  ml  sample  of  venous  blood  was  taken  from  a  vein 
in  the  subject’s  non-dominant  forearm.  Blood  plasma  from  the  sample  was  immediately 
separated  and  frozen  for  subsequent  analyses  of  caffeine,  testosterone  as  a  possible  correlate  to 
target  discrimination,  and  cortisol  as  a  stress  indicator.  Subjects  reported  to  their  designated 
firing  ranges  immediately  after  the  blood  draw. 

Shooting  sessions 

All  ATS  shooting  sessions  were  conducted  using  a  standard  C7A1  assault  rifle  equipped  with  a 
C79  optical  scope.  The  SAT  uses  a  modified  C7A1  that  is  actuated  by  compressed  gas  to 
provide  realistic  recoil  and  synchronized  with  an  amplified  speaker  system  to  simulate  the  rifle 
crack. 

Both  the  ATS  and  SAT  ranges  are  computer  controlled,  allowing  precise  target  appearance  and 
disappearance  scheduling,  as  well  as  detection  of  horizontal-vertical  (x-y)  shot  location.  Whereas 
the  ATS  is  an  outdoor  live-fire  range  employing  pneumatically  lifted  targets  and  an  acoustic 
sensor  array  for  shot  detection,  the  SAT  simulates  the  firing  range  by  projecting  its  image  onto  a 
2.3  x  3.1  m  (7.5  x  10.0  ft)  projector  screen,  and  presenting  target  images  at  programmable  times 
and  locations.  Firing  of  the  SAT  rifle  results  in  laser  light  being  emitted  from  the  rifle  barrel,  and 
its  screen  splash  location  recorded  by  a  hit  detection  camera  for  determination  of  shot  co¬ 
ordinates  relative  to  the  target  center  of  mass. 

Both  FF  and  VIG  sequences  were  duplicated  in  the  ATS  and  SAT  using  a  modified  ‘Figure  E 
Silhouette’  (Cat.  No  1 10-066,  ATA  Defense  Industries,  Camden,  TN)  pop-up  targets  at  a  range  of 
200  m.  The  orientation  of  a  square-wave  grating  pattern  (black  and  white  stripes)  was  used  to 
discriminate  between  friendly  and  foe  targets  (Figure  2).  In  each  case,  targets  were  up  for  4  s  and 
programmed  not  to  fall  if  hit.  Due  to  the  pneumatic  lifting  arm  of  the  ATS,  targets  took 
approximately  1.4  s  to  rise  in  an  arc  from  hidden  horizontal  to  the  fully  vertical  position,  whereas 
SAT  targets  appeared  instantaneously.  Subjects  were  instructed  to  engage  only  foe  targets  with 
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single  well-placed  shots  and  they  were  assigned  non-contiguous  lanes  to  prevent  possible 
feedback  or  interference  from  others  firing  in  parallel  lanes. 


Figure  2.  Modified  ‘Figure  E  Silhouette'  target  with  friendly  (a)  and  foe  (b)  patterns. 


Friend/Foe  discrimination 

During  each  of  the  four  4-min  FF  sequences,  40  targets  (25  foe  and  15  friendly)  were 
programmed  to  appear  at  random  intervals.  A  total  of  four  different  FF  sequences  were  authored 
and  presented  to  all  subjects  in  a  counterbalanced  order  across  the  entire  study.  Subjects  were 
instructed  to  engage  only  foe  targets  and  hold  their  fire  when  a  friendly  target  appeared.  After 
each  sequence,  subjects  were  allowed  a  short  stretch  break  during  shot  scoring  and  system  reset. 


Vigilance 

The  vigilance  task  involved  60  randomly  appearing  foe  targets  over  a  period  of  two  hours. 
Targets  were  randomized  within  four  30-min  sequences  of  15  targets  each.  Due  to  limitations  of 
the  number  of  targets  that  could  be  programmed  in  the  SAT  and  ATS,  this  task  consisted  of  two 
1-h  engagements  interrupted  only  with  a  brief  stretch  break  to  allow  data  downloading  and 
system  reset.  As  the  SAT  has  no  data  downloading  capabilities,  shot  location  had  to  be  recorded 
manually  by  cycling  through  each  shot  and  transcribing  x-y  coordinates  from  the  computer 
screen.  This  process  required  approximately  30  min,  during  which  the  subjects  were  permitted  to 
wait  in  an  adjoining  hallway. 


Questionnaires 

Immediately  prior  to  and  following  the  shooting  session,  subjects  were  asked  to  provide 
subjective  ratings  of  Perceived  Exertion  (RPE)  and  Thermal  Sensation  (TS).  The  Environmental 
Systems  Questionnaire  (ESQ)  was  completed  at  the  beginning  and  end  of  the  experimental 
session,  and  the  Task  Load  Index  (TLX)  at  the  session’s  conclusion. 

The  ESQ  [18]  was  developed  to  assess  symptoms  experienced  by  soldiers  exposed  to 
environmental  extremes,  and  has  been  extended  to  provide  an  indication  of  general  subjective 
perceptions  of  physiological  and  cognitive  strain.  The  ESQ  comprises  68  questions  that  are 
scored  on  a  rank  order  basis  ranging  from  0  (symptoms  not  present)  to  5  (extreme).  Certain  of 
these  questions  may  be  grouped  to  give  additional  sub-indices  of  cold  discomfort,  muscle 
discomfort,  tiredness,  and  well-being. 

The  TLX  [7]  is  a  subjective  workload  assessment  tool  developed  by  the  National  Aeronautics  and 
Space  Administration  (NASA)  to  evaluate  mental,  physical,  and  temporal  demands,  performance, 
effort,  and  frustration.  Subjects  marked  their  ratings  on  a  horizontal  line  that  represented,  from 
left  to  right,  perceptions  of  low  to  high  subjective  workload,  or  in  the  case  of  performance,  from 
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good  to  poor.  The  mark  on  the  line  was  scored  linearly  as  a  percentage  of  the  length  of  the  total 
line. 

Analyses 

Target  detection  was  measured  during  each  4-min  FF  sequence  and  was  based  on  the  number  of 
shots  taken  at  foe  targets  and  shots  withheld  at  friendly  targets.  The  calculated  measures  included 
the  shot-hit  ratio  and  discriminatory  index  (d').  d'  provides  a  measure  of  discrimination 
sensitivity  between  two  similar  alternatives,  and  is  based  on  signal  detection  theory  [15]  using  the 
following  equation: 

d'  =  Z(False  Alarm  Rate)  -  Z(Hit  Rate)  (Eq.  1) 


where 


False  Alarm  Rate  (Noise)  =  Proportion  of  friendly  targets  engaged. 

Hit  Rate  (Signal  +  Noise)  =  Proportion  of  foe  targets  engaged, 

and  Z  represents  the  location  on  a  normal  curve  that  divides  the  area  under  the  curve  into  two 
parts,  depicting  the  hits  and  misses,  or  false  alarms  and  correct  rejections.  By  superimposing  the 
False  Alarm  Rate  and  Hit  Rate  distribution  curves  (normal  distributions  are  assumed),  and 
aligning  the  calculated  Z  scores,  the  difference  between  the  peaks  of  each  curve  is  the  value  d'.  A 
value  greater  than  or  equal  to  1.0  indicates  that  the  subject  was  able  to  reasonably  discriminate 
the  friendly  and  foe  targets. 

Target  engagement  time  (ET)  was  obtained  only  during  ATS  engagements  using  video  and 
auditory  analyses.  A  digital  video  camera  (Model  DCR-TRV315,  Sony  Corp.,  Japan)  was  placed 
adjacent  to  lane  1  and  recorded  target  appearance  and  rifle  shots  at  30  fps.  Only  two  participants 
were  on  the  live  range  during  FF  engagements,  allowing  rifle  recoil  and  exhaust  to  be  clearly 
visible  on  the  videotape.  Frame-by-frame  analysis  was  used  to  determine  the  precise  moment  of 
target  appearance  and  rifle  firing.  Since  up  to  4  participants  would  be  on  the  range  during  VIG 
engagements,  the  limited  field  of  view  of  the  camera  made  it  impossible  to  view  all  participants 
adequately.  As  a  result,  shot  time  was  determined  through  the  analysis  of  the  audio  waveform, 
which  revealed  an  obvious  spike  due  to  the  rifle  crack.  Speed  of  sound  delay  was  corrected 
according  to  the  distance  of  the  participants  from  the  camera  and  ambient  temperature  using  the 
following  equation: 


^  ShOt  ^ 


Mane 


video 


33 1.4 +  0.6 -T- 


(Eq.  2) 


where 

tShot  -  true  time  of  rifle  shot  (s), 

tvideo  -  time  of  rifle  crack,  as  determined  by  audiovisual  analysis  (s), 
diane  =  distance  from  camera  to  marksman  (m),  and 
Ta  =  ambient  temperature  (°C). 


Marksmanship  performance  was  based  on  shot  accuracy  (distance  relative  to  the  target’s  centre  of 
mass)  and  precision  (shot  group  tightness).  Accuracy  was  measured  by  the  constant  error  [CE  - 
distance  between  the  centre  of  impact  (Cl)  and  the  target  centre  of  mass  (CM)]  and  the  shooting 
error  (SE,  defined  by  the  average  distance  of  each  shot  from  CM).  Precision  was  measured 
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according  to  the  mean  shot  radius  (MR,  defined  by  the  average  of  the  straight  line  distance 
between  each  shot  and  Cl),  and  the  area  (AREA)  and  diagonal  (DLA)  of  the  shot  group 
dispersion.  All  measures  of  accuracy  and  precision  were  adopted  from  Johnson  [9]  and  applied  to 
both  FF  and  VIG  in  the  ATS,  and  during  VIG  in  the  SAT.  Due  to  time  limitations,  these  data 
were  not  recorded  during  SAT  FF  sequences.  Marksmanship  data  were  summarized  for  each  4 
min  FF  sequence  and  each  30  min  of  VIG. 

All  target  engagement  and  marksmanship  scores,  TS,  RPE,  and  ESQ  data  were  analyzed  using  a 
3-way  [Range  (2)  x  Drug  (2)  x  Time  (2  or  4)]  analysis  of  variance  (ANOVA),  while  the  TLX  data 
were  analyzed  by  a  2-way  [Range  (2)  x  Drug  (2)]  ANOVA.  The  Neuman-Keuls  post-hoc  test 
was  employed  where  significant  main  effects  were  found  (p  <  .05).  All  statistical  analyses  were 
completed  using  Statistica©  software  (StatSoft  Inc.,  Tulsa,  OK).  Unless  otherwise  stated,  all 
results  are  reported  as  mean  ±  SD. 
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Results 


Blood  Analysis 

Blood  samples  were  obtained  for  all  subjects.  As  anticipated,  plasma  caffeine  levels  were  higher 
during  the  drug  trial  (2098  ±  1908  ng'ml'1)  vs.  the  placebo  trial  (550  ±  710  ngTnl'1)  (p  <  .01). 
There  were  no  differences  in  cortisol  (1 1.55  ±  5.24  vs.  10.75  ±  4.56  |ig/dL)  and  in  total 
testosterone  (4.10  ±  0.86  vs.  4.00  ±  0.94  ng/mL)  levels  between  the  ATS  and  SAT  ranges, 
respectively 

Questionnaires 

RPE  and  TS 

Complete  data  were  available  for  1 1  subjects  for  the  RPE  and  TS  questionnaires  with 
results  shown  in  Figure  3  and  4.  There  was  a  main  effect  of  caffeine  such  that  lower  RPE 
scores,  indicating  less  perceived  effort,  were  noted  in  both  the  ATS  and  SAT  ranges, 
compared  to  the  placebo  condition  (p  <  .05).  A  drug-time  interaction  existed  such  that 
RPE  scores  were  significantly  lower  with  caffeine  at  the  end  of  the  trial  (p  <  .05).  There 
was  no  change  in  RPE  over  time  with  caffeine. 


RPE 


5.0 


ATS-D  ATS-P  SAT-D  SAT-P 


Figure  3.  Mean  values  ±  SD  of  the  RPE  scores  across  all  trial  conditions;  see  text  for  significant  effects. 


Main  effects  for  thermal  sensation  were  found  for  all  conditions  of  range,  drug,  and  time. 
Subjects  reported  significantly  lower  TS  scores  (cooler)  on  the  ATS  range  (p  <  .01), 
when  taking  the  placebo  (p  <.01),  and  by  the  end  of  the  session  (p  <  .05).  Ambient 
temperature  in  the  SAT  facility  was  controlled  to  19  -  21°C,  while  the  outdoor  ATS 


9 


DRDC  Toronto  TR  2003-003 


temperatures  averaged  2.8  ±  3.8°C  for  the  morning  sessions  and  10.8  ±  2.5°C  for  the 
afternoon  sessions. 


Figure  4.  Mean  values  ±  SD  of  the  TS  scores  across  all  trial  conditions;  see  text  for  significant  effects. 


Environmental  Symptoms  Questionnaire  (ESQ) 

Table  2  provides  a  summary  of  the  four  sub  indices  and  individual  questions  where 
significant  effects  were  found.  Subjects  reported  being  less  cold  (p  <  .01)  and  had  a 
greater  sense  of  well-being  (p  <  .05)  on  the  SAT.  Well-being  and  muscle  soreness  were 
both  adversely  affected  (p  <  .05  and  p  <  .01,  respectively)  over  the  duration  of  an 
experimental  session.  Although  there  was  a  main  effect  of  cold  sensation  increasing  over 
time  (p  <  .01),  there  was  an  interaction  with  range  such  that  cold  sensation  increased  by 
the  end  of  the  ATS  while  there  was  no  difference  across  the  SAT  session  (p  <  .01). 

Caffeine  interacted  with  time  with  respect  to  both  sensations  of  tiredness  and  cold. 
Caffeine  consumption  decreased  subjective  sensations  of  tiredness,  while  tiredness 
increased  in  the  placebo  condition  (p  <  .05).  Subjects  taking  the  placebo  felt  colder  at  the 
end  of  the  session  (p  <  .05). 
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Table  2.  Pre  and  post  session  scores  of  the  Environmental  Symptoms  Questionnaire  (ESQ)  where  0  =  no  symptoms  and  5  =  extreme; 

see  text  for  significant  effects. 
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Individual  ESQ  questions  showing  significant  effects  were  centered  on  subjective 
sensations  of  muscular  soreness,  cold,  and  concentration/boredom.  As  would  be 
expected,  bodily  aches,  concentration,  blurred  vision,  restlessness,  boredom,  and 
hunger  worsened  with  time.  Cold  hands  and  feet,  and  chill  were  more  prevalent  on 
the  ATS,  and  increased  with  time,  while  vision  was  blurrier  on  the  SAT  and  worsened 
with  time. 

Caffeine  consumption  resulted  in  the  subject's  hands  feeling  less  cold  with  time,  than 
in  the  placebo  sessions  (p  <  .05).  A  greater  sensation  of  thirst  was  also  noted  with 
caffeine  (p  <  .05),  and  this  effect  interacted  with  time  such  that  thirst  was  higher  prior 
to  the  start  of  the  session,  but  did  not  increase  with  time  (p  <  .05).  The  placebo 
condition  resulted  in  lower,  initial  thirst  scores,  which  increased  to  the  same  level  of 
the  caffeine  condition  post  session  (p  <  .05).  Caffeine  and  time  interacted  such  that 
tiredness  tended  to  increase  with  P  and  decrease  with  D  (p  <  .01).  By  sessions  end, 
tiredness  was  significantly  less  in  the  D  condition. 

Subjects  felt  colder  in  the  hands,  feet,  and  body  on  the  ATS  vs.  SAT,  and  their  hands 
also  felt  colder  with  time.  Complaint  of  blurred  vision  was  greater  in  the  SAT  (p  < 
.05)  and  worsened  with  time  (p  <  .05).  Wakefulness  was  lower  on  the  SAT  (p  <  .05) 
and  degraded  with  time  (p  <  .05). 


The  Task  Load  Index  (TLX)  was  completed  at  the  end  of  each  experimental  session. 
The  only  significant  effect  noted  was  a  decrease  in  perceived  performance  on  the 
SAT  (5.5  ±  2.5  and  5.5  ±  2.8  during  D  and  P,  respectively)  as  compared  to  the  ATS 
(4.0  ±  2.2  and  4.4  ±  2.2)  (p  <  .05). 

Target  detection  and  marksmanship 

Friend/Foe  discrimination 

Due  to  time  constraints  and  technical  limitations,  engagement  times  and 
marksmanship  were  only  obtainable  for  the  ATS,  and  are  reported  elsewhere  [3],  The 
discriminatory  index  (d')  showed  no  difference  across  all  conditions,  and  indicated 
that  the  subjects  were  able  to  reasonably  discriminate  between  friendly  and  foe 
targets.  However,  when  comparing  the  appropriate  and  inappropriate  responses  [i.e., 
engaging  foe  targets  (25  out  of  40)  or  holding  fire  on  friendly  targets  for  a  perfect 
score  of  40],  the  results  revealed  a  range  effect  such  that  more  inappropriate  responses 
occurred  on  the  SAT  (p  <  .05).  Post-hoc  analysis  also  revealed  that  less  correct 
identifications  were  made  during  the  first  FF  sequence  (p  <  .05),  and  there  was  no 
difference  in  the  subsequent  sequences. 

While  the  number  of  shots  taken  were  not  affected  by  the  experimental  condition,  the 
number  of  targets  hit  was  greater  on  the  ATS  (p  <  .01),  as  shown  in  Figure  5.  The 
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ratio  of  targets  hit  to  number  of  shots  taken  was  also  higher  on  the  ATS  (p  <  .01). 
This  difference  was  considerable  with  only  54%  of  SAT  shots  finding  their  mark  vs. 
87%  of  ATS  shots.  ATS  shooting  performance  remained  unchanged  across  the  four 
sequences,  while  SAT  performance  was  worse  during  the  first  sequence  compared  to 
the  following  three  (p  <  .01). 


30.0 


25.0 


12  3  4 

Sequence 


Figure  5.  Comparison  of  number  of  targets  hit  in  each  FF  sequence.;  *  indicates  a  significant  effect 
of  range  (  p<  .01)  and  a  range  x  time  interaction  (p  <  .01). 

Vigilance 


Engagement  time  was  only  available  for  the  ATS  and  are  reported  elsewhere  [3].  More 
targets  where  hit  on  the  ATS  range  (p  <  .01)  and  fewer  shots  were  fired  in  the  SAT  (p  <  .01). 
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The  hit-shot  ratio  was  also  less  on  the  SAT  (p  <  .01),  as  shown  in 


1.20 


1.00 


12  3  4 


Sequence 


Figure  6,  and  subjects  tended  to  take  a  greater  number  of  shots  in  the  caffeine  condition  as 
compared  to  placebo  (p  <  .01). 

-1.20  - - 


1.00 


12  3  4 

Sequence 


Figure  6.  Hit  -  Shot  ratio  comparison  of  live  (ATS)  and  simulated  (SAT)  firing  across  all  VIG 
sequences;  *  indicates  a  significant  effect. 
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Marksmanship  data  generated  in  the  SAT  were  generally  deemed  to  be  of  poor  quality  and 
susceptible  to  technological  deficiencies.  It  was  therefore  necessary  to  filter  the  data  by 
accepting  shots  within  10  cm  of  the  target  for  analyses.  These  deficiencies  and  the  rationale 
for  the  data  criterion  are  described  in  greater  detail  in  the  discussion  section  of  this  report. 
The  amount  of  useable  SAT  marksmanship  data  was  reduced  to  4  subjects. 

Subsequent  analyses  of  the  shot  metrics  indicated  that  precision  and  accuracy  were  different 
on  the  two  ranges.  Where  there  was  a  main  range  effect,  the  scores  indicated  poorer 
performance  on  the  SAT  (Table  3).  Performance  tended  to  decline  with  time,  although  this 
was  not  statistically  significant. 


Table  3.  Marksmanship  during  vigilance  on  the  SAT  resulting  in  a  main  effect  of  range  (p  <  .05).  All 
values  are  in  cm  with  the  exception  of  AREA  which  is  in  cm 2 . 


ATS 

SAT 

■ 

VIG1 

VIG2 

VIG3 

VIG4 

VIG  1 

VIG  2 

VIG  3 

VIG  4 

CE 

21 .3411 1.45 

20.06+11.53 

2019.63 

25.63114.58 

31 .0711 3.72 

30.61113.59 

24.19115.12 

SE 

26.8±9.5 

25.9618.83 

24.22+9.76 

25.219.36 

28.08+15.05 

26.44+18.28 

30.8411 5.51 

26.22+15.17 

MR 

14.12±5.09 

12.7414.2 

12.6+4.29 

14.7515.17 

1 4.6717.95 

10.4418.12 

13.117.16 

13.45+8.57 

AREA 

597.791352.4 

451. 54 1247.04 

619.01+300.94 

773.381460.86 

253.671238.61 

183.34+172.45 

306.37+161.88 

326.23+174.44 

DIA 

59.17+19.61 

55.86+19.68 

52.42118.37 

59.85118.8 

55.18133.02 

40.05+30.85 

47.99+27.62 

49.24+35.19 

Range  and  caffeine  interacted  significantly  with  DIA  (p  <  .05)  and  MR  (p  <  .01).  Caffeine 
did  not  exert  any  influence  on  these  marksmanship  measures  in  the  ATS  condition,  but  tended 
to  decrease  performance  in  the  SAT  (Figure  7). 
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Discussion 


The  results  of  this  study  indicate  that  marksmanship  using  the  FATS  IV  SAT  was  markedly 
different  than  live  fire.  In  almost  every  performance  measure  analyzed,  shooting  in  the 
simulator  produced  poorer  results,  despite  the  limited  validation  of  simulator  use  for  training 
purposes.  Although  the  validation  studies  were  able  to  derive  simple  correlations  between 
live  and  simulated  shooting  scores  on  standardized  firing  relays,  the  dependent  variables  used 
to  test  the  robustness  of  their  conclusions  were  restricted  to  the  skill  level  of  the  marksmen  [2; 
19]  and  level  of  difficulty  of  the  relay  [16].  In  addition,  the  correlations,  while  statistically 
significant,  explained  only  a  small  portion  of  the  performance  variability.  These  validation 
studies  suggest  that  there  may  be  a  limit  to  the  scope  of  conformity  of  performance  between 
the  live  and  simulated  environments.  While  a  main  effect  of  range  type  was  most  common  in 
our  study,  there  were  many  instances  where  changes  in  performance  in  the  simulator  due  to 
the  effect  of  time  or  caffeine  did  not  concur  with  the  live  fire. 

The  most  notable  difference  between  the  two  ranges  was  the  ambient  surrounding.  During  the 
several  days  of  the  study,  outdoor  temperatures  on  the  ATS  ranged  from  approximately  -3  to 
14°C.  Within  each  trial,  there  were  daily  variations  in  wind  speed,  cloud  cover,  and  sun 
angle.  Moreover,  the  final  day  of  testing  was  confounded  by  an  unexpected  snowfall.  In 
contrast,  the  simulated  fire  was  held  in  a  relatively  constant  environment  where  the 
temperature  and  lighting  were  controlled.  The  effects  of  these  weather  variants  were  evident 
in  the  ratings  of  TS  and  ESQ. 

Not  surprisingly,  subjects  felt  cooler  on  the  live  range  with  TS  cold  sensation  increasing  over 
the  duration  of  a  session.  This  may  partially  account  for  the  better  live  fire  performance,  as 
Tikuisis  et  al.  [18]  reported  improved  marksmanship  with  cold  strain,  probably  due  to  an 
increase  in  arousal.  The  cold  sensation  index  of  the  ESQ  mirrored  the  TS  results  with  the 
added  significant  range  x  time  interaction  indicating  no  change  in  cold  sensation  with  time  in 
the  simulator.  However,  cold  sensation  was  attenuated  with  caffeine  ingestion.  Although  no 
physiological  measurements  were  taken  to  determine  whether  body  temperatures  increased, 
recent  investigations  do  not  support  a  thermoregulatory  effect  [18],  suggesting  that  only  the 
perception  of  cold  was  affected. 

All  rifle  firing  was  done  from  the  supported  prone  position.  Over  the  duration  of  a  session, 
subjects  complained  of  physical  and  psychological  fatigue.  Muscle,  back  and  shoulder  aches, 
general  physical  well  being,  concentration,  hunger,  and  boredom  were  all  adversely  affected. 
As  marksmanship  tended  to  degrade  with  time,  it  is  likely  that  postural  and  psychological 
fatigue  were  contributing  factors.  As  the  supported  prone  position  facilitates  the  greatest 
firing  accuracy  due  to  minimal  rifle  movement  and  postural  sway,  as  compared  to  a  sitting, 
kneeling,  or  standing  posture  [20],  it  is  possible  that  marksmanship  degradation  due  to  fatigue 
would  be  greater  in  a  less  stable  posture.  Occasional  stretch  breaks  or  changes  in  posture 
might  be  considered  when  operational  conditions  permit.  Caffeine  appeared  to  provide  some 
relief  to  the  sensation  of  tiredness  and  perceived  exertion  over  time.  This  is  not  surprising 
given  its  well-documented  effects  as  a  stimulant  [4].  Thirst  was  also  greater  with  caffeine 
consumption,  which  concurs  with  Smit  and  Rogers  [17]  who  reported  such  an  increase  in  low 
habitual  users. 
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In  the  simulator,  subjects  indicated  a  higher  sense  of  well-being  and  decreased  wakefulness. 
These  finding  are  not  surprising  given  that  the  facility  housing  the  simulator  was  controlled  at 
a  comfortable  ambient  temperature  and  invariant  to  outdoor  conditions  (eg.,  wind,  cloud,  etc.). 
Paradoxically,  the  comfortable  conditions  may  have  contributed  to  the  subjects’  poorer 
marksmanship  during  the  vigilance  task.  Tikuisis  et  al.  [18]  found  marksmanship 
improvements  when  subjects  were  exposed  to  a  disturbing,  but  not  debilitating  level  of  hot  or 
cold  strain,  as  compared  to  a  thermoneutral  baseline  condition.  It  was  postulated  that  the 
thermal  strain  caused  a  beneficial  arousal  such  that  the  subjects  were  better  able  to  focus  on 
the  shooting  task.  Additionally,  light  levels  in  the  simulator  were  kept  low  to  facilitate 
viewing  of  the  projected  display  and  correct  laser  detection.  Being  devoid  of  the  natural 
stimulation  of  the  sights  and  sounds  of  an  outdoor  environment,  and  lying  on  a  padded  firing 
platform,  the  subjects  often  struggled  to  remain  alert.  A  possible  recommendation  to  counter 
these  lulling  effects  would  be  to  add  ambient,  range,  and/or  battlefield  sounds,  to  ensure 
adequate  room  ventilation,  and  to  avoid  an  overly  comfortable  platform. 

Subjects  also  complained  that  their  vision  was  blurry  in  the  SAT.  Although  light  levels  were 
not  measured  in  this  study,  a  combination  of  low  room  illumination  and  screen  luminance, 
limited  image  resolution,  and  focusing  difficulties  with  the  red,  blue,  and  green  light  channels 
of  the  projection  system  contributed  to  this  impaired  visual  acuity.  Since  the  completion  of 
this  study,  upgrades  to  the  SAT  have  included  the  addition  of  a  single  lens  projection  system 
that  appears  to  provide  for  much  greater  luminance  and  clarity.  A  second  factor  contributing 
to  the  poor  image  quality  was  the  use  of  a  standard  C79  scope.  While  the  targets  were 
situated  at  a  simulated  distance  of  200  m,  the  actual  screen  upon  which  targets  were  displayed 
was  only  8  m  from  the  subject,  well  within  the  focal  length  of  the  scope.  A  SAT-specific 
scope,  with  appropriate  focal  length,  has  since  been  made  available  and  should  improve 
visibility  in  future  studies. 

Decreased  wakefulness  and  visual  acuity  may  also  be  responsible  for  the  subjects’  perception 
of  lower  performance  on  the  SAT.  These  findings  were  similar  to  those  reported  by  Gula  [5] 
in  addition  to  comments  regarding  the  obtrusive  placement  of  the  compressed  air  hose  on  the 
hand  guard.  Numerous  weapons  stoppages,  as  well  as  the  system  failing  to  register  the 
weapon’s  zeroing  information,  led  to  a  high  degree  of  frustration.  The  net  result  of  these 
irritants  decreased  the  subject’s  confidence  in  the  system  and  may  have  negatively  affected 
their  motivation  and  performance. 

Analysis  of  d'  (discriminatory  index)  indicated  that  the  subjects  were  able  to  adequately 
discriminate  between  the  friendly  and  foe  targets,  although  the  number  of  correct 
identifications  in  the  SAT  were  less  than  on  the  live  range.  Again,  some  subjects  remarked 
that,  due  to  the  poor  resolution  of  the  video  display,  the  friend/foe  markings  on  the  targets 
appeared  pixilated  and  more  difficult  to  distinguish  than  on  the  live  range.  A  decrease  in 
resolution  affects  the  texture  gradient,  causing  SAT  targets  to  appear  farther  away.  This 
might  partially  explain  the  poor  shooting  performance  on  the  SAT  and  the  findings  of  Gula 
[5]  who  noted  SAT  performance  being  better  than  live  at  100  m,  but  significantly  degraded  at 
200  m.  Additional  perceptual  cues  such  as  relative  size,  cast  shadows,  and  other  pictorial 
features  may  have  been  inappropriate  or  missing  in  the  authored  engagements  resulting  in 
errors  of  target  distance  judgement.  As  the  first  FF  sequence  resulted  in  the  poorest 
performance,  it  is  possible  that  a  period  of  adjustment  and  familiarization  was  required  before 
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the  subjects  could  adequately  engage  the  targets,  despite  being  practiced  during  the  pre-trial 
familarization  session. 

In  addition,  fewer  SAT  targets  were  engaged  in  the  placebo  condition  and  in  the  latter 
sequences.  By  extension,  this  concurs  with  the  findings  of  Johnson  and  Murello  [13],  who 
reported  that  approximately  10%  of  all  shots  were  fired  after  the  target  had  disappeared  after  6 
s  of  exposure.  It  is  possible  that  the  shorter  4  s  target  exposure  time  in  our  study  posed  a 
greater  challenge  to  the  subjects’  vigilance  and  target  engagement,  thereby  allowing  caffeine 
to  exert  its  stimulatory  effect. 

Vigilance  performance  was  also  poorer  on  the  SAT.  Unfortunately,  the  poor  quality  of  data 
restricted  complete  and  reliable  analyses  to  4  subjects.  It  was  felt  that  such  an  analysis  could 
be  justified,  as  the  effects  of  any  system  irregularities  would  be  minimized.  This  re-analysis 
had  little  effect  on  the  overall  interpretation  of  marksmanship  scores  during  the  vigilance  task. 
SAT  scores  were  still  substantially  poorer  than  ATS  scores,  and  marksmanship  still  degraded 
with  time. 

It  was  discovered,  after  the  completion  of  the  study,  that  the  then  current  version  of  the  SAT 
software  occasionally  failed  to  register  the  zeroing  information  of  the  weapons  (FATS  INC. 
have  since  verified  the  existence  of  this  deficiency).  This  helps  explain  the  biased  or  erratic 
shooting  displayed  in  certain  sequences.  An  example  of  this  is  shown  in  Figure  8,  which 
demonstrates  a  distinct  bias  in  a  subject’s  shooting  between  two  contiguous  sequences  within 
a  session.  Gula  [5]  using  the  previous  generation  SAT  (FATS  III)  system  noted  a  similar 
phenomenon.  In  some  circumstances,  where  initial  weapon  and  sight  alignment  was  good, 
and  little  zero  adjustment  was  necessary,  failure  to  register  a  weapon  properly  would  be 
impossible  to  detect.  A  recent  upgrade  to  the  SAT  software  has  addressed  this  issue  and,  if 
successful,  should  make  the  SAT  a  more  reliable  simulator. 
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Figure  8.  Shot  location  and  centre  of  impact  (Cl)  of  two  contiguous  vigilance  sequences  for  one 

subject. 

A  second  possible  explanation  for  the  difference  in  scores  is  the  range  in  which  the  SAT  and 
ATS  registered  shot  location.  The  ATS  handled  low  shots  differently  from  SAT,  as  low  shots 
would  either  hit  the  ground  and  not  be  registered,  or  ricochet  and  tumble,  hitting  the  target  at  a 
location  higher  than  their  original  trajectory  would  have  resulted.  On  the  other  hand,  shots 
as  low  as  1  m  below  the  target  were  registered  by  the  SAT  without  being  considered  to  have 
landed  in  front  the  target.  The  net  result  of  these  discrepancies  would  be  a  lower  vertical  bias 
to  SAT  marksmanship  metrics. 
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Conclusion 


A  comparison  of  SAT  and  live-fire  marksmanship,  with  and  without  caffeine  consumption,  in 
either  friend/foe  or  vigilance  engagements  revealed  substantial  differences  in  subjective 
perceptions  of  the  environment  and  shooting  performance.  Many  of  these  differences  can  be 
attributed  to  the  conditions  of  the  ranges  and  the  quality  of  the  SAT  software  and  projection 
system.  Despite  an  attempt  to  create  similar  environments  and  target  engagements,  the  results 
of  this  study  indicate  that  performance  on  the  SAT  was  poorer  than  live  fire.  Differences  in 
weather/climate  control  conditions,  arousal,  visual  acuity,  and  system  reliability  affected  not 
only  shooting  performance,  but  on  how  caffeine  influenced  target  detection  and 
marksmanship  on  each  range.  Unfortunately,  the  colder  condition  on  the  ATS  range 
interacted  with  the  subjects’  performance  and  disallowed  a  fair  validation  of  the  SAT  as  a 
surrogate  research  platform  for  studies  involving  the  effects  of  stressors/aids  on  TD&M. 
However,  had  the  ambient  conditions  been  similar  between  the  two  ranges,  it  is  doubtful  that 
results  on  the  present  SAT  would  have  matched  those  on  the  ATS  due  to  the  technical 
limitations  of  the  SAT. 

Measurements  of  shot  location  on  the  SAT  were  deemed  to  be  of  poor  quality  and  required 
considerable  care  when  the  results  were  analyzed.  Extraction  of  the  data  was  a  tedious  and 
time-consuming  process,  which  disrupted  the  continuity  of  the  experiment  and  created 
undesirable  downtime  for  the  subjects.  They  eventually  became  frustrated  and  disenchanted 
by  the  system’s  cumbersome  operation  and  unreliability,  which  undoubtedly  contributed 
further  to  their  poor  performance.  Since  this  study  has  been  completed,  a  new  generation  of 
SAT  software  and  hardware  has  been  made  available.  Several  of  the  most  significant  issues, 
such  as  visual  acuity,  weapon  function,  and  zeroing  registration  have  been  addressed.  With 
these  system  improvements,  coupled  with  improved  data  reporting  capabilities  and  a  greater 
effort  to  create  a  SAT  facility  that  provides  a  similar  level  of  arousal  as  a  live  range,  the  SAT 
should  realize  its  potential  of  providing  a  cost-effective  and  flexible  marksman  research 
platform. 
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List  of  symbols/abbreviations/acronyms/initialisms 


DND 

Department  of  National  Defence 

SAT 

Small  Arms  Trainer 

ATS 

Automated  Targetry  System 

FF 

Friend-Foe 

VIG 

Vigilance 

P 

Placebo 

D 

Drug  (caffeine) 

CE 

Constant  Error 

SE 

Standard  Error 

MR 

Mean  Radius 

DIA 

Diameter 
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